Linguistic Issues in Grace (Evaluation of Part-of-Speech Tagging for French)

GRACE is the first large-scale evaluation program of taggers for French. This experiment allowed to compare the assignments of Parts-of-Speech tags by various different taggers, on a common corpus of literary and journalistic texts. The evaluation relied on the acceptance by the participants of a reference formalism for morpho-syntactic description (the reference tagset) used by an expert to tag the evaluation corpus, and by the participants to provide a description (mapping table) of their own tagset. The global strategy was to make the reference tagging and tokenization of the finest grain possible. The reference tags were decomposed in Parts of Speech (main category) and lists of additional attributes, thus defining detailed syntactic patterns. The steps of the GRACE program are described, and the main adjudication issues are reported. The linguistic issues encountered during this experiment were linked to the difficulty to project relevant information about the sentence structure at the token level. Information derived from local analysis may be accepted as well as information derived from a larger context. Although the limitations of the experiment are acknowledged, the GRACE program proved to offer interesting opportunities to assess the state of the art in PoS tagging.